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"You got to know when to hold em, know when to fold em, know when to walk away.. 
-Kenny Rogers 



Supporting Software: This article is accompanied by Maple packages ChowRobbins, 
STAD JE, and WALKSab, and Mathcmatica packages Builder . m (and notebook Builder . nb) 
as well as STADJE.m, available from the webpage of this article 
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1. When to Stop? 

In a delightful and insightful recent "general" article [4], the great probabilist 
and master expositor Theodore Hill described, amongst numerous other intriguing 
things, a more than forty-year-old open problem, due to Y.H. Chow and Herbert 
Robbins [2] that goes as follows: 

Toss a fair coin repeatedly and stop whenever you want, receiving as a reward 
the average number of heads accrued at the time you stop. If your first toss is a 
head, and you stop, your reward is 1 Krugerrand. Since you can never have more 
than 100 percent heads, it is clearly optimal to stop in that case. If the first toss is 
a tail, on the other hand, it is clearly best not to stop, since your reward would be 
zero... 

Then Ted Hill goes on to comment that if the first toss is a tail and the second is a 
head, then it is good to go, since by the law of large numbers, you would eventually 
do (at least slightly) better than one half. [It turns out that in this case of one 
head and one tail, the expected gain of continuing the game is larger than 0.6181]. 

Hill further claims that it is optimal to stop if the initial sequence is tail-head- 
head. [This is wrong. It turns out, thanks to our computations, that it is optimal 
to go, and the expected gain is > 0.6693 rather than 2/3.] 

The exact stopping rule, i.e. the decision whether to stop or go, is still an open 
problem for (infinitely) many cases. As we will see, it is easy (with computers!) 
to prove that it is optimal to go for many cases where this is indeed the case, but 
proving rigorously that for a given position it is optimal to stop is a challenging, still 
open, problem. It is analogous to disproving vs. proving a mathematical conjecture. 
To disprove it, all you need is to come- up with a specific counterexample, whereas 
to prove it, you need to show that no counterexample exists. 

l 
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2. The Continuous Limit 



Way back in the mid sixties, this problem was tackled by such luminaries as Chow 
and Robbins themselves [2J, Aryeh Dvoretzky [3], and Larry Shepp [BJ. Chow and 
Robbins proved the existence of a stopping sequence, (3 n , such that you stop as soon 
the number of heads minus the number of tails, after n tosses, is > (3 n . While Chow 
and Robbins only proved the existence of the "stopping sequence" , Dvoretsky [3] 
proved that f3 n /^/n lies between two constants, for n sufficiently large, while Larry 
Shepp [BJ went further and proved that 



exists and equals 0.83992 . . . , a root of a certain transcendental equation. 

But this beautiful work, like most of "modern" probability theory, is asymptotic, 
talking about large n. It tells us nothing, for example, about the still open (3s (pre- 
sumably 2) and not even about /3ioo- For example, the still-open question whether 
(3$ — 2 can be phrased as follows. 

If currently you have five heads and three tails, should you stop ? 

If you stop, you can definitely collect 5/8 = 0.625, whereas if you keep going, your 
expected gain is > 0.6235, but no one currently knows to prove that it would not 
eventually exceeds 5/8 (even though this seems very unlikely, judging by numerical 
heuristics). 

3. The Role of Computers in Pure Mathematical Research 

We really enjoyed Hill's fascinating article, but we beg to differ on one (impor- 
tant!) issue. Hill ([3], p. 131) claims that: 

"Computers were not useful for solving that problem. In fact, all the problems de- 
scribed in this article were solved using traditional mathematicians' tools-working 
example after example with paper and pencil; settling the case for two, three, and 
then four unknowns; looking for patterns; waiting for the necessary Aha! insights; 
and then searching for formal proofs in each step. " 

So far, this is all factual, so there is nothing to disagree with. Ted Hill was 
merely describing how he and his colleagues do research in pure mathematics. But 
then came an opinion that we do not agree with: 

"Computers are very helpful for after-the-fact applications of many results, such as 
backward induction. But in theoretical probability, computers often do not signifi- 
cantly aid the discovery process. " 

This may have been true in the past, and to a large extent still at present, but 
we believe that in the future computers will be more and more useful even-and 
perhaps especially-in theory, since in addition to their obvious role as number- 
crunchers, they are also starting to do a great job as symbol-crunchers, and 
even as idea-crunchers. One recent example is [IT], and the present article is 
another illustration, even though we do quite a bit of number-crunching as well. 



(2.1) 
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4. The Backward Induction Algorithm 

The reason that it is so hard to decide (in some cases, for example with 5 
heads and 3 tails) whether to stop (and collect, for sure, the current number of 
heads divided by the current number of tosses [i.e. hj(h + t)]), or to keep going, 
(expecting to do better), is the somewhat unrealistic assumption that we live for 
ever. Since in real life, we eventually would have to quit playing after N tosses, for 
some finite N, and collect whatever we get then. So let's consider the bounded case 
where the number of coin-tosses is < N, for a fixed, possibly large, yet finite N. 
Compromising however with our immortality fantasy, we will let the player collect 
1/2, once reaching the JV-th coin toss, if the number of tails exceeds the number 
of heads, citing the law of large numbers that "guarantees" that "eventually" we 
will be able to (at least) break even. In other words, we let people who die in debt 
take advantage of the law of large numbers down in hell. [It turns out that, as far 
as the soon-to-be-defined limit, F(h,t) goes, one does not need this assumption, 
and it is possible to insist that the player collects h/N no matter what, but the 
breaking-even assumption considerably accelerates the convergence.] 

Let's call /jv(/i,i) the expected pay-off in this bounded game, if you currently 
have h heads and t tails. Following Chow and Robbins, there is a simple backward 
induction (dynamical programming) algorithm for computing /jv(/i, i) for all (h,t) 
with h + t < N. 



Boundary conditions: when h + t = N: 

(4.1) f N (h,N-h)=max(l/2,h/N) , (0 < h < N). 

Backward Induction: 

f N (h+l,t)+f N (h,t+l) h 



(4.2) fwiKt) = max 



h + t 



[If you keep going, the expected gain is [/at(/i + 1, t) + /jv(/i, t + l)]/2, if you stop 
the expected (and actual) gain is h/(h + t). ] 

[/at(/i, i) is implemented in procedure CR(h,t,N) in ChowRobbins. CRm(h,t,N) 
is a faster version] . 

It is obvious that, for each specific h and t, /at(/i, t) is an increasing sequence in 
N , bounded above by 1, so we know that the limit 

(4.3) F{h,t) := lim f N (h,t) , 

"exists" . 

Fantasizing that we actually know the values of F(h, t), (as opposed to knowing 
that they "exist"), we can decide whether to stop or go. If F(h,t) = h/(h + t) 
then we stop, and otherwise we go. This assumes that the player merely evaluates 
situations by expectation. As we know from the St. Petersburg paradox, expectation 
is not everything, and a player may choose to guarantee collecting h/(h + t) rather 
than taking a huge chance of eventually getting less. We will later describe other 
criteria for stopping. 

Julian Wiseman estimates F(0, 0) to be 0.79295350640 .... 

The difficulty in proving, for a given number of heads and tails, (h, t), that it is 
optimal to stop is that we need rigorous non-trivial (i.e. < 1) upper bounds valid 
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for /zv(/i> t) for all TV. Then this would also be true of F(h, t), the limit as TV — * oo 
of fpf(h,t). On the other hand it is easy to come up with lower bounds, namely 
fN (h, t) is < /jv(/i, t) for all TV > TVo, so in particular every specific /jv (h, t) serves 
as a lower bound of F(h, t), so it follows that whenever, for some Nq, it is true that 
h/(h + t) < fN (h, t), then we know for sure that it is good to go. 

5. The (probable) sequence /3 n 

So let's be realistic and take N to be 50000, rather than oo. The sequence 
/3„(50000), that we conjecture equals the "real thing" f3 n = /3„(oo), for 1 < n < 185, 
equals: 

I, 2, 3, 2, 3, 2, 3, 2, 3, 4, 3, 4, 3, 4, 3, 4, 5, 4, 5, 4, 5, 4, 5, 4, 5, 4, 5, 4, 5, 6, 5, 6, 5, 6, 5, 6, 5, 6, 

5, 6, 5, 6, 7, 6, 7, 6, 7, 6, 7, 6, 7, 6, 7, 6, 7, 6, 7, 6, 7, 8, 7, 8, 7, 8, 7, 8, 7, 8, 7, 8, 7, 8, 7, 8, 7, 8, 
7, 8, 9, 8, 9, 8, 9, 8, 9, 8, 9, 8, 9, 8, 9, 8, 9, 8, 9, 8, 9, 8, 9, 8, 9, 10, 9, 10, 9, 10, 9, 10, 9, 10, 9, 10, 
9,10,9,10,9,10,9,10,9,10,9,10,9,10,9,10,11,10,11,10,11,10,11,10,11,10,11,10, 

II, 10, 11, 10, 11, 10, 11, 10, 11, 10, 11, 10, 11, 10, 11, 12, 11, 12, 11, 12, 11, 12, 11, 12, 11, 12, 
11, 12, 11, 12, 11, 12, 11, 12, 11, 12, 11, 12, 11, 12, 11, 12, 11, 12, 11. 

We observe that for 1 < n < 9, (3 n 2 = n while for 10 < n < 13, it equals n — 2. 
This seems to be in harmony with Shepp's theorem, even for small n. 

6. The question of when to stop and when to go depends on how 

long you expect to live 

We mentioned above that Ted Hill [4] erroneously stated that 2 heads and 1 tails 
is a stop. Well, he was not completely wrong. With N < 50, in other words, if the 
game lasts at most 50 rounds, and as soon as you have tossed the coin 50 times you 
must collect max(l/2, h/50), then (2, 1) is indeed a stop. However, if the duration 
of the game is > 51, then it becomes a go. We say that the cutoff for (2, 1) is 51. In 
the following list, the i-th. item is a pair. Its first component is that position with 
h + t = i that has the largest h for which (h, t) is a go (for N = 2000, and most 
probably (but unprovably) for N = oo). Its second component is the smallest N 
for which it stops being stop and starts being go. Notice the cautionary tales of 
the position with 10 heads and 7 tails that only starts being a go with N = 1421, 
and the position with 24 heads and 19 tails, for which N = 1679 is the start of 
go-dom. 

Here is the list of pairs: 



[[[0,1], 2], [[1,1], 3], [[2,1], 51], [[2, 2], 5], [[3, 2], 7], [[3, 3], 7], [[4, 3], 9], [[4, 4], 9], [[5, 4], 11], 
[[6, 4], 35], [[6, 5], 13], [[7, 5], 23], [[7, 6], 15], [[8, 6], 21], [[8, 7], 17], [[9, 7], 21], [[10, 7], 1421], 
[[10, 8], 23], [[11, 8], 91], [[11, 9], 25], [[12, 9], 57], [[12, 10], 25], [[13, 10], 47], [[13, 11], 27], 
[[14, 11], 43], [[14, 12], 29], [[15, 12], 43], [[15, 13], 31], [[16, 13], 43], [[17, 13], 277], [[17, 14], 43], 
[[18, 14], 139], [[18, 15], 43], [[19, 15], 103], [[19, 16], 45], [[20, 16], 87], [[20, 17], 45], [[21, 17], 79], 
[[21, 18], 47], [[22, 18], 75], [[22, 19], 49], [[23, 19], 73], [[24, 19], 1679], [[24, 20], 71], [[25, 20], 423], 
[[25, 21], 71], [[26, 21], 249], [[26, 22], 69], [[27, 22], 185], [[27, 23], 69], [[28, 23], 155], [[28, 24], 71], 
[[29, 24], 137], [[29, 25], 71], [[30, 25], 125], [[30, 26], 73], [[31, 26], 119], [[31, 27], 73], [[32, 27], 113], 
[[32, 28], 75], [[33, 28], 109], [[34, 28], 833], [[34, 29], 107], [[35, 29], 477], [[35, 30], 107], [[36, 30], 343], 
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[[36, 31], 105], [[37, 31], 275], [[37, 32], 105], [[38, 32], 235], [[38, 33], 105], [[39, 33], 211], [[39, 34], 105] 
[[40, 34], 193], [[40, 35], 105], [[41, 35], 181], [[41, 36], 105], [[42, 36], 171], [[42, 37], 105], [[43, 37], 165] 
[[43, 38], 107], [[44, 38], 159], [[45, 38], 1039], [[45, 39], 155], [[46, 39], 679], [[46, 40], 153], 
[[47, 40], 513], [[47, 41], 151], [[48, 41], 419], [[48,42], 149], [[49, 42], 361], [[49, 43], 147], [[50, 43], 321] 

[[50,44], 147], [[51, 44], 293], [[51, 45], 147], [[52, 45], 271], [[52,46], 145], [[53, 46], 255], 
[[53, 47], 145]]. 

7. More Statistical Information 

The above strategy for deciding when to stop is entirely based on expectation. 
Even if we pursue this strategy, it would be nice to have more detailed information, 
like the standard deviation, skewness, kurtosis and even higher moments. Ideally, 
we would like to know the full probability distribution. 

Let's call GN(h,t;x) the fractional polynomial in the variable x (i.e. a lin- 
ear combination of powers x a with a rational numbers) such that the coeff. of 
x a is the probability of getting exactly a as pay-off in our game, still pursuing 
the strategy of maximizing the expected gain. Of course Gjv(/i, i;l) = 1 and 

-j-Gjv(M;2)U = i = fN{h,t). We have: 
ax 

Boundary conditions: when h + t = N: 

(7.1) G N (h,N-h;x) =i max(1/2 ' W (0<h<N) . 
Backward Induction: 

(x h/(h+t) , if (h,t) is STOP 

(7.2) G N (h, t;x) = l G N (h + 1, t; x) + G N (h,t + 1; x) jf ^ ^ j s qq ' 

[GN(h,t; x) is implemented in procedure CRt(h,t,N,x) in ChowRobbins.] 

Once we have Gjv(/i, t; x), we can easily get all the desired statistical information. 

8. Another Way to Gamble 

In real life we don't always want to maximize our expected gain. Often we have 
a certain goal, let's call it g, and achieving or exceeding it means everlasting happi- 
ness, while getting something less would mean eternal misery. In that case we need 
a different gambling strategy, that is really straightforward. Keep playing until 
h/(h + t) > g, and if and when you reach it, stop. Otherwise keep going to the 
end, until h + 1 = N . In that case, of course, the stop states are those for which 
h/(h + t) > g. It is still of interest to to know what is the probability of happiness. 
Let's call this quantity P/v(<7; h, t). We obviously have: 



Boundary conditions: when h + t = N: 
(8.1) P N (g;h,N-h) = \°' 



if h/N < g 
if h/N > g. ' 



(i 
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Backward Induction: When h + t < N, Pat (3; h,t) equals 1 if h/(h + t) > g while 
it equals (Pisr(g; h + 1, t) + P/v(g; h,t+ l))/2 otherwise. 

We leave it to the reader to formulate the backward induction scheme for finding 
the probability generating function for the present strategy. 

9. Comparative Gambling 

Let's compare the two strategies using both criteria. Of course the first one 
always is better in the maximum expectation category and the second is always 
better in maximizing the probability of achieving the goal. 

With N = 200, at the very beginning, your expected gain, under the first way 
is 0.7916879464, but your probability 

• of getting > 0.6 is 0.6917238235 (the second way gives you probability 
0.7753928313, but your expected gain is only 0.6742902054) 

• of getting > 0.7 is 0.5625000000 (the second way gives you probability 
0.6075176458, but your expected gain is only 0.5787939263) 

Much more data can be found by using procedure SipurCG in the Maple package 
ChowRobbins, and posted in the webpage of this article. 

10. Probabilities of Escape 

The second strategy gives rise to the following interesting computational ques- 
tion: 

Fix a > b > 1 relatively prime. What is the probability that the number of heads 
divided by the number of tails 

(i) will ever exceed a/bl 

(ii) will either exceed or be equal to a/bl 

This question was raised and answered by Wolfgang Statdje [8] who proved that 
this quantity is a root of a certain algebraic equation. A related problem is treated 
by Nadeau [5]. 

Stadje's result can also be deduced from the more general treatment by Ayyer 
and Zcilbcrgcr |T|, that contains a Maple package that automatically derives the 
algebraic equation for any general set of steps. For practical purposes, however, 
we found it easiest to compute these probabilities directly, in terms of the discrete 
functions W(x,y) and W s (x, y) that count the number of lattice walks from the 
origin to (x,y) staying in the required region. This is contained in the Maple 
package STADJE. 

Here is some data gotten from STADJE. The numbers below answer questions (i) 
and (ii) above, respectively, for each of the listed pairs (a, b). 

(a,b) = (2,1) : 0.6180339887,0.6909830056 ; 
(a, b) = (3,1) : 0.5436890127,0.5803566224 ; 
(a,b) = (3,2) : 0.7481518342,0.7754441182; 
(a,b) = (4,1) : 0.5187900637,0.5362190123 ; 
(0,6) = (4,3) : 0.8091410707,0.8229424412; 
(a,b) = (5,1) : 0.5086603916,0.5170258817; 
(a, b) = (5,2) : 0.5876238826,0.5996923731; 
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(a, b) = (5,3) : 0.7158769909,0.7276461121; 
(a,b) = (5,4) : 0.8453136528,0.8534748833; 

Also of interest is the sequence enumerating the number of walks, staying in the 
region y > a/bx, from the origin to a point of the form (n, n), whose asymptotics 
can be proved to be of the form Ci(a, 6)4™/-^, for some constant Ci(a, 6), and the 
sequence enumerating the number of walks, still staying in the same region, ending 
at (an, bn), whose asymptotics has the form C 2 (a, b)((a + b) a+b / (a a b b )) n /n 3/2 . The 
Maple package STADJE (and Mathematica package STADJE.m) computes any desired 
number of terms, and estimates Ci(a, b), 62(0,6). The webpage of this article 
contains some sample output. 



11. From Number-Crunching to Symbol Crunching 

So far, we have designed numerical computer programs whose outputs were 
numbers. But what about closed form? It would be too much to hope for an 
explicit formula for fpj{h,t) valid for arbitrary N, h, t, but, with experimental-yet- 
rigorous mathematics, we can find explicit expressions, as rational junctions in n 
for 

(11.1) f 2n+ i(n + a,n- a-m + 1), 

where n and m are positive integers and a is an integer. 
Let 

(11.2) F(m, a, n) = /2n+i(?7 + a, n — a — m + 1) 

for n, m, and a as before. Since h + t < 2n + 1, then F(m, a, n) are values below 
the topmost diagonal on the backward induction triangle. 

Some values of F(m,a,n) are not hard to get. For instance, the value of 
F(m, a, n), for a > 1 and 1 < m < 2n, is given by 

(11.3) F(m,a,n) 



2n — ?7i+l 

whereas the value of F(m, a, n), for a < —m and 1 < m < 2n, is given by 
(11.4) F(m,a,n) = i 

Both formulas can be proved by induction. Hence, we arc reduced to finding for- 
mulas for F(m, a, n) when —m < a < 1. 

Our first approach is to make the computer conjecture closed forms for F(m, a, n). 
For this, we programmed a Mathematica function called GF [this function can be 
found in the webpage of this article]. It takes as input a positive integer m and 
two variables n and a, and another positive integer bound. Here, the computer 
makes the assumption that n > bound. For the guessing part, GF uses the auxiliary 
function GuessRationalFunction. This procedure is similar to GuessRat, which 
accompanied the article [7] and can be found in [10]. The output of GF, which is 
the guess formula for F(m, a,n), is a piecewise rational function of n with m + 2 
pieces. 
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Example 11.1. For m 



(11.5) 



2 and n > 3, GF conjectures 

a < -2 

9n + 2 



F(2,a,n) 



1/2 
8ra- 



16n 

8n 2 - 



16n 2 
n + a 



8n 



a = 
a > 1 



2n- 1 

We point out that formulas conjectured by GF only work for n sufficiently large. 
In fact, empirical evidence suggests that the bound on n grows exponentially in m 
i.e. as we go down on the backward induction triangle, the bound for which the 
formulas are valid grows exponentially As a result, these formulas are not directly 
useful for determining stop vs. go status. 

It is possible to study the recursion formula of f n {h,t) to get explicit formulas 
for F(m, a. 2n + 1). For example, a simple analysis gives 



(11.6) 



F(l,a,2n+ 1) = 



1/2 
4n - 



8n- 
n 4 



a 



2n 



which is true for n > 1, and 



(11.7) 



1/2 

8n + 5 



F(2,a,2n+ 1) = < 



16n 

8n 2 - 



a < -1 
a = 

a > 1 

a < -2 
a = —1 



9n + 2 



16n 2 
n + a 



8n 



o = 
a > 1 



2n-l 

which is true for n > 3. However, these calculations become tedious rapidly. 

To our surprise, it turns out that Mathcmatica, via the built-in functions Assuming 
and Refine, is able to handle these recursions and get the desired formulas. We 
programmed a Mathcmatica function called BUILDER, whose input is an integer m 
and two variables n and a. BUILDER calculates closed-form formulas for F{m, n, a) 
and provides the smallest n where they start to hold. For instance, 



(11.8) F(5,a,2n+1) 



1/2 
64n 



128n 
32n 2 - 



33 

- 20n + 1 



64n 2 + 32n 
64n 3 + 30n 2 - 



13n - 3 



128n 3 - 
64n 4 + 8n 3 



32n 
- 46n 2 



5n + 3 



128n 4 - 128n 3 
256n 5 - 124n 4 



32ti 2 4 
340n 3 



32n 
h91n 2 



75n — 6 



512n 5 

n + a 



1280n 4 + 640n 3 + 320n 2 - 192n 



2n-4 

was calculated by BUILDER and holds for n > 102. 



a < -5 
a = -4 

a = —3 

a = -2 

a = -1 

a = 

a > 1 
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The starting places, for n, where the formulas of F(m, a, n) begin to hold, with 
1 < m < 16, are: 1, 3, 12, 37, 102, 263, 648, 1545, 3594, 8203, 18444, 40973, 
90126, 196623, 426000, and 917521 respectively. These values seems to satisfy the 
recurrence defined by 

a\ = 1 

a m = 2a m _i + r m valid for m > 1, 

where r m is given by 

n = 

r 2 = 1 
r 3 = 6 

r rn = 2r m _i + m — 3 valid for m > 3. 

We are pleased to report that the formulas conjectured by GF and the ones found 
by BUILDER agree. 
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